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Abstract 

Background: Many candidate vaccine strategies against human immunodeficiency virus (HIV) infection are under 
study, but their clinical development is lengthy and iterative. To accelerate HIV vaccine development optimised trial 
designs are needed. We propose a randomised multi-arm phase l/ll design for early stage development of several 
vaccine strategies, aiming at rapidly discarding those that are unsafe or non-immunogenic. 

Methods: We explored early stage designs to evaluate both the safety and the immunogenicity of four 
heterologous prime-boost HIV vaccine strategies in parallel. One of the vaccines used as a prime and boost in the 
different strategies (vaccine 1) has yet to be tested in humans, thus requiring a phase I safety evaluation. However, 
its toxicity risk is considered minimal based on data from similar vaccines. We newly adapted a randomised phase II 
trial by integrating an early safety decision rule, emulating that of a phase I study. We evaluated the operating 
characteristics of the proposed design in simulation studies with either a fixed-sample frequentist or a continuous 
Bayesian safety decision rule and projected timelines for the trial. 

Results: We propose a randomised four-arm phase l/ll design with two independent binary endpoints for safety 
and immunogenicity. Immunogenicity evaluation at trial end is based on a single-stage Fleming design per arm, 
comparing the observed proportion of responders in an immunogenicity screening assay to an unacceptably low 
proportion, without direct comparisons between arms. Randomisation limits heterogeneity in volunteer characteristics 
between arms. To avoid exposure of additional participants to an unsafe vaccine during the vaccine boost phase, an 
early safety decision rule is imposed on the arm starting with vaccine 1 injections. In simulations of the design with 
either decision rule, the risks of erroneous conclusions were controlled <15%. Flexibility in trial conduct is greater with 
the continuous Bayesian rule. A 12-month gain in timelines is expected by this optimised design. Other existing designs 
such as bivariate or seamless phase l/ll designs did not offer a clear-cut alternative. 

Conclusions: By combining phase I and phase II evaluations in a multi-arm trial, the proposed optimised design allows 
for accelerating early stage clinical development of HIV vaccine strategies. 
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Background 

Although several promising approaches to prevent hu- 
man immunodeficiency virus (HIV) infection have been 
put forth in the recent years [1-4], the development of a 
prophylactic vaccine strategy remains a key goal in the 
effort to end the HIV epidemic. 

The main prophylactic HIV vaccine candidates currently 
under study include subunit or epitope-based vaccines 
with or without adjuvants, recombinant virus-vector 
vaccines and deoxyribonucleic acid (DNA) vaccines. 
The combination of different vaccines into heterologous 
prime-boost strategies is today considered a promising 
approach. Indeed, results of a phase IIB trial in Thailand 
(RV144 trial), combining a virus-vector vaccine prime 
with a subunit boost, showed a modest protective effect 
of the tested HIV vaccine strategy [5]. HIV vaccine de- 
velopment remains ongoing and too lengthy. For in- 
stance, clinical development of the components of the 
HIV vaccine strategy evaluated in the RV144 trial 
started in the mid-1990s in Thailand, however the re- 
sults of the aforementioned phase IIB trial only became 
available in 2009. According to development timelines, 
a confirmatory phase III trial with an improved vaccine 
strategy is not projected to start before 2019 [6,7]. 

The fact that immunological correlates of protection 
are not well understood has hampered HIV vaccine de- 
velopment to date [8]. A protective vaccine effect can 
currently not be predicted on the basis of vaccine- 
induced immunogenicity markers. Since transition from 
phase I/II to phase IIB/III trials can thus not rely on a 
validated surrogate immunogenicity endpoint, and large 
sample sizes and resources are required for later-stage 
trials with HIV-acquisition endpoints (phase IIB/III), the 
clinical development of HIV candidate vaccine strategies 
is laborious. Phase I and II HIV vaccine trials usually in- 
clude extensive immunogenicity assessments, measuring 
both the humoral and cellular immune responses to vac- 
cines with different markers and techniques. All of the 
above complicates the decision to set up a large-scale 
IIB/III trial, powered for an HIV-acquisition endpoint. 
These constraints have resulted in only six HIV vaccine 
phase IIB/III trials conducted so far, five of which 
yielded disappointing results without any evidence for 
protective vaccine effects [9-11]. 

Recently, calls for accelerated clinical development of 
HIV vaccine strategies have been put forward, advocat- 
ing the implementation of adaptive trial designs for this 
purpose [12,13]. Indeed, numerous adaptive or multi- 
stage clinical trial designs for different phases of clinical 
development are available in the methodological litera- 
ture. However, these designs were mainly devised for 
cancer treatment trials, which differ from HIV vaccine 
trials. In oncology, the patients who are enrolled in trials 
suffer severe illness requiring treatment. Inefficacious 



strategies must thus be identified as soon as possible to 
avoid harm. In contrast to oncology trials, including tri- 
als for therapeutic cancer vaccines [14], the development 
of prophylactic HIV vaccines targets a non-ill population 
and thus necessitates specific design considerations. 
Moreover, assessment of endpoints is clearer in oncology 
trials both because of a higher incidence and more well- 
defined surrogate endpoints, such as tumour response 
criteria [15]. The application of adaptive trial designs to 
HIV vaccine research is challenging given that numerous 
immunogenicity endpoints without a definite hierarchy 
are evaluated in phase II trials and that clinical develop- 
ment plans should remain flexible enough to accommodate 
new scientific knowledge. Thus, trial design optimisation 
has rarely been considered in HIV vaccine research, yet as 
the number of potential HIV vaccine strategies under de- 
velopment increases it has become of greater interest [16]. 
Gilbert et al. proposed an adaptive phase IIB/III design 
with an HIV-acquisition endpoint for mid- to late-stage 
clinical development (phase IIB/III), which has yet to be 
applied in practice [17]. For early-stage development, 
Moodie et al. recommended a phase IB selection design in 
2006 to prioritise among different candidate vaccine 
strategies the one with the top-rank based on an im- 
munogenicity endpoint. At that time, they considered 
that knowledge about HIV vaccine immunogenicity was 
too limited to select vaccine strategies using bounds based 
on a theoretical response rate [18]. To date, although un- 
derstanding of vaccine-elicited immune responses still re- 
quires much more progress, this justification has evolved. 
We now consider it purposeful to define an insufficient re- 
sponse level in a widely used, validated immunogenicity 
screening assay [19]. 

Currently, more than 20 'generic' HIV candidate vac- 
cines are in early clinical development [16]. Given that 
generic candidates may be tested with different antigen 
inserts, that combinations of different candidate vaccines 
are evaluated and that strategies with the same combin- 
ation may differ in terms of the timing of injections, the 
number of potential vaccine strategies is large. Efficient 
screening of potential strategies and decision making in 
early stage development is thus crucial. In the present 
article, we present an early-stage trial design, integrating 
phases I and II to evaluate both safety and immunogen- 
icity endpoints of several HIV vaccine strategies in paral- 
lel and allowing for the rapid cessation of those with 
insufficient safety or immunogenicity levels. 

Methods 

Motivating example 

Our search for an optimised phase I/II HIV vaccine trial de- 
sign was motivated by the clinical development plan of a 
vaccine portfolio comprising three prophylactic HIV candi- 
date vaccines, referred to herein as Vaccine 1,' '2' and '3'. 
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Vaccine 1 is a viral vector (modified vaccinia Ankara 
(MVA) virus) expressing HIV antigens, which has not 
yet been evaluated in humans with the specific HIV in- 
serts used. It therefore requires a phase I safety study be- 
fore it can be administered to a larger number of 
volunteers. However, the MVA vector with different HIV 
and non-HIV inserts has already been studied in previ- 
ous trials and has a good safety profile in humans 
[20-26]. Minimal toxicity is thus expected for vaccine 1. 
The tested target dose (lxlO 8 plaque-forming units) corre- 
sponds to the standard dose tested for MVA vaccinations 
against smallpox [22,27] and to the dose selected in previ- 
ous dose-escalation studies of MVA vectors with HIV and 
non-HIV inserts [20,24,28,29]. No dose escalation is thus 
planned for this vaccine in our development plan. 

Vaccine 2 is a lipopeptide vaccine, including an equal- 
weight mix of five synthetic HIV-1 peptides coupled to a 
palmytoil tail (HIV LIPO-5). Vaccine 3 is a deoxyribo- 
nucleic acid (DNA) vaccine, encoding for a multi-HIV 
antigen. Both vaccines 2 and 3 have been studied previ- 
ously in phase I and II trials and have a good safety rec- 
ord [30-33]. 

In the clinical development plan of this vaccine portfo- 
lio, the three vaccines are combined into four heterol- 
ogous (that is, using different vaccines in the prime and 
the boost phase) prime-boost strategies (Figure 1). Vac- 
cine 3 is currently being considered only for use as a 
prime vaccine component, and not as a boost [34] . 

In our clinical development plan, the aim of the phase 
I evaluation of vaccine 1 at the target dose is to stop ad- 
ministrations of this vaccine as soon as possible in case 



of a safety problem. The early stage of the phase II 
evaluation aims at screening each of the four prime- 
boost combinations for a minimum immunogenicity 
level. Non- immunogenic or unsafe vaccine strategies 
should be discarded as soon as possible. A 'classical' clin- 
ical development scheme with several phase I and II tri- 
als to address these objectives would be a lengthy 
process. In order to accelerate the clinical development 
and given the high likelihood of a good safety profile of 
vaccine 1 similar to the one observed with different HIV 
inserts, we sought to set up an optimised trial design 
combining phases I and II for evaluation of the four 
prime-boost strategies in parallel. 

For the purpose of the present work, we defined an 
optimised design' as a trial design allowing for a gain in 
timelines (and, if possible, in the number of trial partici- 
pants) compared to separate phase I and II trials, while 
remaining feasible and respecting the requirements out- 
lined below. 

Requirements for the optimised phase l/ll vaccine trial 
design 

In searching for an optimised design appropriate for our 
development plan, we imposed the following methodo- 
logical requirements: 

• First, a randomised multiple arm design is desirable 
to ensure comparability of populations and unbiased 
assessment of several vaccine strategies in parallel, 
even if no direct statistical comparisons are made 
between arms [35]. 
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Figure 1 Overview of the proposed multi-arm phase l/ll design. Legend: W = week. 
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• Second, the design requires an evaluation of both 
immunogenicity and safety endpoints, assessed at 
different time points, allowing for an early safety 
decision rule to emulate the phase I evaluation of 
vaccine 1 during the vaccine prime phase. The dose used 
for vaccine 1 is considered well below the toxic dose. 

• Third, endpoints for immunogenicity and safety are 
not necessarily correlated, since severe vaccine- 
related toxicity events and immunogenicity are likely 
to be unrelated [36]. 

• Fourth, an important constraint of the design is the 
fact that safety and immunogenicity outcomes are not 
observed simultaneously, since the immunogenicity 
measurements of all participants are planned in batch 
at a central laboratory at the end of the trial to allow 
for blinded assessments and to limit measurement 
error due to time and batch effects. 

• Fifth, data integration of phase I into phase II 
evaluation should be possible to allow for an optimal 
use of the collected data. 

• Sixth, the trial design must be compatible with high 
early accrual dynamics and with conduct as a multi- 
centre trial. Prophylactic HIV vaccine trials enrol 
healthy volunteers, a process that allows for better 
planning and management of recruitment dynamics 
than the enrolment of ill patients. To keep costs 
low, a single call for volunteers is planned for our 
trial. As previous vaccine trial recruitment studies 
have shown that a waiting time' between the first 
contact and enrolment may lead to a loss of interest 
in participation [37], the time between the call for 
volunteers and the start of the trial will be kept 
short. Altogether, this is expected to result in high 
early accrual dynamics. 

• Lastly, a design without a control group was 
favoured since HIV-specific immune responses 
can be considered to be close to zero in healthy 
volunteers at low risk of HIV infection, as attested 
by immune response measurements in prophylactic 
HIV vaccine trials at baseline or in a placebo arm 
[20,21,38-40]. A vaccine strategy meeting the 
immunogenicity endpoint can therefore be deemed 
clearly distinct from a potential control group. Given 
that the volunteers in the planned trial will have a 
low-risk profile (restricted by the eligibility criteria) 
and the fact that all participants will receive an 
experimental vaccine strategy, we considered it 
unlikely that differential risk behaviour during the trial 
would have an impact on the immunogenicity endpoint. 

A summary of these requirements is shown in the list 
below: 

■ randomized multi-arm trial 



■ evaluation of both efficacy and safety endpoints, with: 

- early safety decision rule 

- difference in timing of the evaluation between the 
two endpoints 

- assumption that toxicity events and efficacy are 
likely unrelated 

- efficacy outcome not observed in real-time 
(measured in batch in central lab at end of the trial) 

- data integration of phase I into phase II efficacy 
evaluation 

■ compatibility with high early accrual dynamics and 
with conduct as a multi-centre trial 

■ no control arm 

Moreover, although not a formal design requirement, an 
open-label design was preferred since the immunogenicity 
endpoint can be measured objectively and laboratory staff 
performing the measurements centrally will be blinded to 
trial arm and time point. Although a blinded control 
group would limit bias in the evaluation of safety, imple- 
mentation of blinding is difficult in practice: indeed, blind- 
ing would require the introduction of dummy vaccine 
injections in each trial arm, since vaccination time points 
during the prime phase vary between vaccine strategies 
(Figure 1). Additional immunogenicity sampling time 
points after each dummy shot would then be necessary for 
maintaining blinding, requiring large amounts of blood. 
Blinding participants and site staff was thus not pursued 
for this trial design, but we acknowledge that a blinded 
placebo control arm could be helpful for safety assess- 
ments when feasible. 

Search for the optimised design 

We first searched the literature for existing trial designs 
fulfilling the requirements outlined above in order to be 
appropriate for application to early stage clinical HIV vac- 
cine development. We performed a literature search in 
Pubmed/Medline using the search terms ('Clinical Trials, 
Phase II as Topic' [MAJR]) AND ('Therapeutics/adverse 
effects' [MeSH] OR safety OR toxicity). In addition, the re- 
view and manual of phase II designs by Brown et al. were 
used to identify potential designs for our context [41]. 
Moreover, we reviewed the reference lists of the most rele- 
vant articles. 

We then adapted a randomised phase II design to in- 
clude an early safety decision rule during the prime phase 
of vaccine 1. We evaluated two types of statistical ap- 
proaches for the safety decision rule: (1) a fixed-sample 
rule from a frequentist perspective; and (2) a continuous 
safety monitoring approach from a Bayesian perspective. 
Simulation studies were built to evaluate the statistical 
properties of the design with either decision rule. For each 
simulation scenario under different underlying safety and 
immunogenicity hypotheses, the participant outcomes for 
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a trial arm with a safety decision rule (arm 1 in our trial 
design) were drawn from independent binomial distribu- 
tions of the binary safety and immunogenicity endpoints 
respectively, and repeated in 10,000 simulation runs. We 
also performed ancillary simulation studies based on 
draws of participant outcomes from multinomial distribu- 
tions, with specified joint probabilities of the two end- 
points (corresponding to positive correlations from 0.1 to 
0.5 between immunogenicity response and occurrence of 
vaccine-related toxicity events) [42], while keeping the mar- 
ginal proportions of the endpoints constant. Recruitment 
dynamics were not taken into account for the simulation 
studies. Simulations were performed with R software, ver- 
sion 2.13.0 (The R Foundation, Vienna, Austria). 

Furthermore, we projected the expected dynamics of 
recruitment into the trial and the gain in timelines of 
the clinical development plan by the proposed design 
based on our previous experience setting up and con- 
ducting HIV vaccine trials in France [32,43,44] . 

Results and discussion 

We adapted a randomised phase II design by including 
two different types of early safety decision rules. We first 
describe the characteristics of the proposed design with 
each safety decision rule and its practical implications. We 
then discuss potential alternative designs as well as poten- 
tial limitations of our design and further perspectives. 

Characteristics of the proposed optimised phase l/ll 
design 

We propose a randomised four-arm phase I/II design with 
two independent binary endpoints for safety and immuno- 
genicity, including a safety decision rule for vaccine 1 dur- 
ing the prime vaccination phase in arm 1 (Figure 1). The 
design is optimised in terms of a seamless transition from 
phase I to phase II and allows for a parallel, unbiased 
evaluation of several vaccine strategies in phase II. 

The proportion of immunogenicity responders in each 
arm is assessed by a standardised interferon-gamma (IFN-y) 
enzyme-linked immunospot (ELISPOT) assay at week 30, 
that is, 2 weeks after the last vaccine immunisation. There is 
a broad scientific consensus that a successful vaccine to pre- 
vent HIV transmission must be able to elicit both HIV- 
specific T-cells and antibodies responses [45,46] . Although 
previous research has shown that a positive IFN-y ELI- 
SPOT response does not necessarily predict a protective 
vaccine effect against HIV acquisition [47], it is assumed 
that the IFN-y ELISPOT response is at least a marker that 
the vaccine has some effect on the T-cells of the immune 
system. Under this assumption, the IFN-y ELISPOT 
should allow to screen for vaccine immunogenicity and 
therefore seems adequate as a criterion to identify 
non-immunogenic vaccine strategies that do not elicit 



any cellular response [19]. Additional immunogenicity 
markers will be assessed as secondary endpoints. 

Each arm of the phase II design has a single-stage 
Fleming design for the binary immunogenicity endpoint 
[48], comparing the proportion P of participants with a 
positive IFN-y ELISPOT response to a theoretical un- 
acceptable proportion of 50%. The randomised design 
with parallel arms is thus not intended for formal 
between-arm comparisons but to limit heterogeneity in 
volunteer characteristics between arms. In contrast with 
the randomised phase II selection designs used in oncol- 
ogy [14], this design does not aim to select one single 
strategy (that is, the 'best' strategy) at the end of the 
phase II design but retains all strategies with responses 
above a minimal level. The decision rule is formulated 
per arm and does not depend on the number of parallel 
arms in the trial, which distinguishes its operating char- 
acteristics from those of randomised phase II selection 
designs [49]. The sample size of n = 23 per arm was de- 
termined from published sample size tables based on 
exact binomial distributions, with 90% power, one-sided 
type I error rate of 5%, pi of 80% (target proportion of 
IFN-y ELISPOT responders), and pO of 50% (unaccept- 
ably low proportion of IFN-y ELISPOT responders) [50]. 

The integrated phase I evaluation of vaccine 1 is based 
on an independent binary safety endpoint at week 2 in 
arm 1, evaluating the proportion of participants without 
any grade 3 or 4 clinical or biological adverse events re- 
lated to vaccine 1 immunisation, reported from week 0 
to week 2. Event relatedness will be reviewed and grades 
validated by the Endpoint Review Committee. 

We evaluated two types of safety decision rules, with 
the aim to avoid exposure of additional participants to 
an unsafe vaccine during the vaccine boost phase. 

First, we considered a frequentist fixed-sample ap- 
proach, which was also derived from a single-stage 
Fleming design, with 90% power, one-sided type I error 
rate of 5%, pi of 95% (target proportion of participants 
without any vaccine-related adverse event), and pO of 
70% (unacceptably low proportion of participants with- 
out any vaccine-related adverse event). Based on the 
proportion at which the lower bound of the one-sided 
exact 95% confidence interval includes pO, this translates 
into the following safety decision rule for vaccine 1 [50]: 
if more than two out of the first 19 participants in arm 1 
experience a vaccine-related adverse event by week 2, 
then all vaccine 1 injections will be stopped. 

Second, we studied a Bayesian sequential safety moni- 
toring approach, allowing (1) for formally incorporating 
the previously available knowledge about the safety of 
MVA vectors in the prior distribution, and (2) for con- 
tinuously updating the posterior distribution after each 
participant or observed event during the trial [51]. We 
used an enthusiastic prior distribution (Beta (6, 0.3)) to 
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reflect the sound prior knowledge about the safety of 
this vector type. The Bayesian safety decision rule was 
then defined based on the posterior probability that the 
proportion of participants without any vaccine-related 
adverse event is below the target proportion of 0.95. At 
any sequential analysis, the injections of vaccine 1 would 
be stopped if this posterior probability was greater than 
95% (Prob (Psafe < 0.95 1 data) > 0.95). 

Given the planned rapid accrual dynamics of the 
trial, the proposed safety decision rules do not aim at 
halting enrolment, which may be completed before 
the decision rule applies. Thus, the decision rule is 
not necessarily expected to impact the number of vol- 
unteers enrolled. Rather, it aims at taking a go/no-go 
safety decision before the start of the vaccination 
boost phase at week 20 to avoid exposing additional 
participants to vaccine 1 in the other trial arms. If ad- 
ministrations of vaccine 1 are stopped, arms 1 and 2 
are halted but arms 3 and 4 continue, with partici- 
pants in arm 4 receiving boost injections of vaccine 2 
instead of vaccine 1. 

Simulation results for each of the two safety decision 
rules are shown in Table 1. 

Table 2 summarises the combined operating character- 
istics of a trial arm in the final phase I/II design, includ- 
ing both the frequentist analysis of the immunogenicity 
endpoint at week 30 as well as the continuous Bayesian 
monitoring of the safety endpoint at week 2. The simu- 
lated operating characteristics were satisfactory under all 
combinations of the hypotheses for safety and immuno- 
genicity respectively, with a maximum overall error rate 
of 12.1%. 

Equivalent simulations of the combined operating 
characteristics of the design including the frequentist 
fixed-sample safety decision rule showed relatively 
similar results, albeit with a slightly higher maximum 
overall error rate (overall probability of erroneous 
conclusion 14.2% for the scenario with simulated 
safety and efficacy proportions of 0.95 and 0.80, 
respectively). 



Additional simulations for scenarios with an inter- 
mediate safety proportion of 80% and the fixed- 
sample frequentist decision rule or the continuous 
Bayesian decision rule showed probabilities of carry- 
ing the vaccine forward to the next step of the devel- 
opment plan in between those observed in the 
simulations with a safety level of 70% and 95%, 
respectively. 

Introducing slight to moderate correlations between 
immunogenicity response at week 30 and vaccine- 
related adverse events at week 2 in the simulations did 
not change the operating characteristics. This is due to 
the fact that the endpoints are based on the marginal 
proportions, which were kept constant by design of the 
simulations. 

Practical implications of the proposed design 

As illustrated in Figure 2, compared with separate phase 
I and II trials, the proposed design with either safety de- 
cision rule allows for an estimated gain of approximately 
12 months in the timelines of the clinical development 
plan. 

In terms of trial logistics and data management, to be 
able to apply either of the decision rules in a timely 
manner, the proposed design requires real-time monitor- 
ing and validation of vaccine-related grade 3 and 4 ad- 
verse events until week 2. 

The frequentist fixed-sample safety decision rule is 
stringent but requires high early accrual dynamics in order 
to have 19 evaluable participants at week 2 in arm 1 before 
any other participant reaches week 20 (first boost injection 
of vaccine 1). By using a continuous monitoring approach 
after each participant, the Bayesian rule allows for more 
flexibility and can be applied even if accrual dynamics de- 
viate from the initial projections. This may for instance be 
the case if a certain time lag between the enrolment of the 
first couple of volunteers is desired in order to not expose 
several persons at once to a vaccine being first adminis- 
tered in human. Moreover, the Bayesian method allows 
for a direct interpretation of posterior probabilities. 



Table 1 Comparison of operating characteristics of the two types of safety decision rules 



Type of safety decision rule 




Simulation scenario 




Vaccine safe (P sa f e = 0.95) 


Vaccine unsafe (P safe = 0.7) 




Probability of stopping 


Probability of stopping Participant number at stop (median; IQR) 


Fixed-sample frequentist rule at n = 19 


0.07 


0.95 19 (1 9-1 9) a 


Continuous Bayesian monitoring 


0.05 


0.96 8 (4-12) 



Fixed-sample frequentist rule: Interim analysis after 19 participants. Vaccine considered unsafe at interim if lower bound of one-sided exact 95% confidence inter- 
val of observed proportion <0.7 for safety endpoint. 

a ln a strict application of the fixed sample-design the decision rule only applies when the outcome of 19 participants has been observed. If the application of the 
rule was handled more flexibly (that is stopping as soon as a third participant experiences an vaccine-related grade 3 or 4 adverse event, regardless of the denom- 
inator), stopping would occur after nine participants in median (IQR 6-12). 

Continuous Bayesian monitoring of the safety endpoint: Sequential analysis after each participant or event. Vaccine considered unsafe at the first analysis where 
the posterior probability that the vaccine is below the target level (Psafe <0.95) exceeds 95%. Enthusiastic prior: beta (6,0.3). 
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Table 2 Combined operating characteristics of the design including a frequentist efficacy evaluation and the 
continuous Bayesian safety decision rule 



Simulation scenario 


Proportion of outcomes in 10,000 simulations of one trial arm (%) 


Safety hypothesis 


Immunogenicity hypothesis 


Vaccinations 
stopped 
(unsafe) 


Vaccinations not 
stopped (safe), 
inefficacious at final 


Vaccinations not 
stopped (safe), 
efficacious at final 


Overall probability 
of erroneous 
conclusion 


Unsafe (0.70) 


Inefficacious (0.50) 


95.5 


4.2 


0.3 


4.5 


Unsafe (0.70) 


Efficacious (0.80) 


95.6 


0.3 


4.1 


4.7 


Safe (0.95) 


Inefficacious (0.50) 


4.8 


91.4 


3.8 


8.6 


Safe (0.95) 


Efficacious (0.80) 


5.3 


6.8 


87.9 


12.1 



Simulation methods: Simulation of independent binomial distributions for safety and efficacy, respectively. Simulation of participant outcomes for one trial arm 
with continuous Bayesian monitoring of the safety endpoint, repeated in 10,000 trial simulations per simulation scenario. 

Bayesian continuous monitoring of the safety endpoint. Decision rule for safety outcome: Vaccine considered unsafe at the first analysis where the posterior probability 
that the vaccine is below the target level (Psafe <0.95) exceeds 95%. Enthusiastic prior: beta (6,0.3). 

Fixed-sample frequentist analysis of immunogenicity endpoint with 23 participants: conclusion that vaccine strategy is efficacious at final analysis if lower bound 
of one-sided exact 95% confidence interval of observed proportion >0.5. Proportion of trials with erroneous outcomes shown in bold. The overall risk of erroneous 
conclusion corresponds to the sum of erroneous trial outcomes. 



Given the early safety decision rule, the standard fre- 
quentist estimate of the proportion of participants with 
severe vaccine-related adverse events throughout the 
whole follow-up in the trial (week 30), planned as a sec- 
ondary safety endpoint, may be biased in the final ana- 
lyses. If the final analysis stage is done from a frequentist 
perspective, specific methods for an unbiased estimation 
will thus need to be considered in order to take into ac- 
count the fact that the final estimation is conditional on 
not stopping before [52]. 



Alternative existing designs 

Most alternative phase I or II designs available in the lit- 
erature were not further considered due to the specific 
requirements for our trial design (see list of design re- 
quirements above and Table 3). 

We gave thorough consideration to alternative designs 
to evaluate the safety of vaccine 1 within a phase II trial 
but ruled them out for the following reasons: 

Continuous toxicity monitoring approaches from 
the frequentist perspective, such as those suggested by 
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Figure 2 Gain in timelines with optimised phase l/ll design compared to separate phase I and II trials. 
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Table 3 Frequentist early-stage designs described in the literature and requirements for the vaccine trial design 

Relevant features of the vaccine trial 
Trial design and endpoints 



Logistics 



Absence of Efficacy No control arm Possibility to move 

a validated outcome not with 'standard' more than one 

surrogate endpoint assessed in strategy or strategy to the next 

for efficacy 3 real-time b placebo development stage 



Safety and High early Randomised 

efficacy endpoints accrual multi-arm 

at different dynamics d multi-center 

time points c trial 



Potential alternative frequentist designs 



Type 



Example 



A) Designs for both efficacy and toxicity evaluation 

Bryant and Day design [53] 



Non-comparative bivariate 
two-stage designs 

Seamless phase l/l I design 



Design proposed by Messer et al. 
(including a 3 + 3 design for the 
integrated phase I evaluation) [54] 



B) Toxicity stopping rules integrated in efficacy designs 



Non-comparative stopping rule based 
on continuous toxicity monitoring per 
serious adverse event 

Non-comparative stopping rule based 
on continuous toxicity monitoring per 
participant 

Non-comparative stopping rule based 
on group-sequential approach 



Continuous monitoring proposed 
by Kramar et al. [55] 

Continuous monitoring proposed 
by Ivanova et al. [56] 

Probabilistic approach proposed 
by Yu et al. [57] 



C) Designs for efficacy evaluation, considered in combination 
with toxicity stopping rules in B) 



Non-comparative two-stage or 
multi-stage designs 

Non-comparative treatment selection 
design 

Comparative multi-arm designs 



Gehan's, Simon's or Fleming's 
design [48,58,59] 

Ranking design by Simon [35] 



Comparative phase II designs; 
screening designs [60] 

Group sequential designs; adaptive 
designs with comparative decision 
rule [61] 



No 



No 



No 



No 
No 
No 



No 



No 



No 



No 
No 



No 



No 



No 



No 



No 



No 



Non-exhaustive list of trial features and alternative frequentist designs. Only main features of vaccine trial leading to incompatibility with alternative designs are indicated. 

No: characteristic of the vaccine trial not compatible with alternative design. 

The term 'non-comparative' is used to indicate that no inter-arm comparison is required. 

a No single validated endpoint in HIV vaccine immunogenicity trial, since correlates of vaccine protection are unknown. Currently, multiple different immunogenicity measurements are of interest without any definite 
hierarchical order in their relevance (multidimensional data) and no obvious definition of a composite endpoint. In the present vaccine trial, the primary immunogenicity endpoint is only used as a screening assay to 
discard out non-immunogenic strategies. 

immunogenicity measurements done in batch on frozen samples at the end of the trial. 
c Safety evaluation at week 2; Efficacy evaluation at week 30. 

d High early accrual dynamics expected after a single call for volunteers in the media. 
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Goldman et al, Kramar et al, Ivanova et al. or Ray et al 
[42,55,56,62] require that the participants (or events) 
be observed sequentially (that is, one by one), which 
is not granted with high accrual dynamics. An alter- 
native approach to Bayesian continuous safety moni- 
toring could have been designed based on the 
predictive probability instead of the posterior distri- 
bution. In the context of efficacy monitoring with 
slightly higher maximum sample sizes than in our de- 
sign it has been shown to have similar operating 
characteristics than the method based on the poster- 
ior distribution [63]. However, applying decision rules 
based on future probabilities to a safety endpoint is 
conceptually less intuitive, and we believe that a rule 
based on the current posterior distribution better re- 
sembles the decision making process for safety. For 
the same rationale, we did not pursue the approach 
published by Yu et al. who suggested toxicity moni- 
toring based on a probabilistic approach estimating 
the future rate of toxicity [57]. 

Most existing bivariate two-stage designs for com- 
bined safety and efficacy evaluation, be they frequen- 
tist or Bayesian [53,64-67], were developed in the 
context of cytotoxic cancer drug development and 
generally require both the efficacy and the safety out- 
come to be observed and analysed with the same tim- 
ing. This is not the case in our vaccine trial where 
the immunogenicity outcome is measured in batch at 
the end of the trial. Although a group-sequential bi- 
variate design with increased flexibility, accommodat- 
ing different schedules for efficacy and safety analyses 
was also suggested, this design still requires at least 
one analysis time including both safety and efficacy 
endpoints [68]. 

Lastly, we considered the seamless phase I/II frequen- 
tist design proposed by Messer et al. [54], the rationale 
of which is very close to ours. However, their design is 
appropriate for a single-arm trial, using a 3 + 3 enrol- 
ment scheme in phase I, but is too complex to imple- 
ment in the context of a multi-center multi-arm trial 
and is not compatible with the accrual dynamics ex- 
pected for the healthy volunteers in our trial. Other 
seamless phase I/II designs comprising a dose-finding 
phase have the same limitations for application in our 
context [69]. 

Limitations of the proposed design 

The design we propose fulfils all our specified require- 
ments but has some limitations. 

First, the set-up of a combined phase I/II design is 
resource-efficient under the assumption that vaccine 1 is 
safe, but is less efficient if vaccine 1 injections were 
stopped for insufficient safety. However, even if this is 
the case, two arms will still continue the trial (arms 3 



and 4, then both receiving the same vaccine strategy 
with vaccine 2 as boost injections and merged together 
for the final analysis). This will double the number of 
participants receiving the strategy with vaccine 3 prime and 
vaccine 2 boost and result in increased precision for the 
non-comparative statistical analysis of this strategy. Thus, 
even if vaccine 1 is stopped, at least some resource- 
efficiency of the proposed four-arm trial will be maintained. 
However, the proposed design would be less efficient in a 
context, in which the safety record of the vaccine candidate 
is more uncertain. 

Second, the proposed design does not include a fur- 
ther transition into the next development phases, and a 
separate phase II follow-up trial will be necessary to 
evaluate the immunogenicity spectrum of the vaccine 
strategies with more precision. 

Third, albeit randomised, our trial is not designed for 
inter-arm comparisons. Investigators should be careful 
about making indirect comparisons of the point esti- 
mates in the different arms without considering preci- 
sion [70]. 

Lastly, no multiplicity adjustment is planned in our 
design to account for the multiple trial arms. However, 
the control of false-positive selection for immunogenicity 
(type I error) is not a major concern in this early-stage 
trial, since independent follow-up trials are scheduled in 
the clinical development plan. 

Further perspectives in the clinical development plan 

Set-up of a HIV vaccine trial using the design with the 
Bayesian safety decision rule and including a 24-hour lag 
between the enrolment of the first 20 volunteers is cur- 
rently ongoing in a protocol of the French Vaccine Re- 
search Institute (VRI). 

At the end of this trial it is planned to carry all im- 
munogenic and safe vaccine strategies forward to further 
evaluations in another phase II immunogenicity trial, 
since more data should be collected before deciding to 
set up a large scale phase IIB/III trial with an HIV- 
acquisition endpoint. However, unless the knowledge 
about immunological correlates of protection and surro- 
gate makers evolves substantially, it will remain difficult 
to predict protective vaccine efficacy on the basis of 
phase II immunogenicity results and the decision to 
move the clinical development to a phase IIB/III trial 
will be delicate. A group-sequential phase IIB/III design 
could thus be envisaged, with interim looks at the HIV- 
acquisition endpoint to allow for early stopping of strat- 
egies for lack of efficacy or harm [17]. 

Conclusions 

We used a pragmatic approach to design an optimised 
randomised phase I/II trial for the evaluation of the 
safety and immunogenicity of several HIV vaccine 
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strategies. The design can include either a frequentist 
fixed-sample or Bayesian continuous early safety deci- 
sion rule and allows for data integration of phases I and 
II for the final analysis. Therefore, acceleration of early 
clinical development of HIV vaccine strategies is, to 
some extent, possible, but requires thoughtful planning 
at the design stage. Validation of surrogate markers for 
HIV vaccine efficacy will be crucial for the implementa- 
tion of more complex adaptive phase II designs. 

During the HIV vaccine development process, candi- 
date vaccines are often improved in terms of the antigen 
inserts used. The situation where a well-known vector 
with a modified insert is administered first in humans 
without dose escalation is thus not unusual in HIV vac- 
cine development portfolios. Similar thoughts may also 
apply to vaccine research for other diseases requiring 
complex vaccine strategies, for example, malaria vac- 
cines. Although some aspects discussed in this article 
are specific to HIV vaccine research, the suggested early- 
phase design could therefore also be useful for the clin- 
ical development of other complex vaccine portfolios 
with similar methodological requirements. 
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